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Abstract.  We  report  on  our  numerical  results  concerning  the  capacity  of 
certain  binary  symmetric  channels  with  asymmetric  erasures.  Plots  of  our 
numerical  capacity  solutions  are  offered  in  contrast  to  some  of  the  existing 
literature. 


1.  Introduction 

We  report  on  some  numerical  discrepancies  in  the  plots  from  [9]  that  were  noted 
by  Daniel  Kang  when  he  interned  at  the  Naval  Research  Laboratory  as  a  SEAP 
student  during  the  summer  of  2009.  In  addition,  the  seeming  impossibility  of  ob¬ 
taining  closed  form  solutions  for  the  channel  capacity  in  this  paper  has  served  as 
motivation  for  the  capacity  bounds  illustrated  in  [7].  The  results  in  this  report 
were  obtained  by  using  the  Blahut-Arimoto  algorithm  for  numerically  calculating 
the  capacity  [4]. 


2.  Optical  Fiber  Communication 


Figure  1:  Channel  transition  diagram  for  BAC/AE  channel 


Manuscript  approved  December  17,  2009. 


l 


2 


IRA  S.  MOSKOWITZ  AND  DANIEL  D.  KANG 


Most  of  this  section  is  a  description  of  what  is  given  in  [9].  Therefore,  we  will 
freely  paraphrase  the  theoretical  background  as  given  in  [9] ,  with  the  understanding 
that  this  section  borrows  heavily,  without  quotation  marks  (for  better  readability), 
what  is  given  in  [9].  Additional  information  on  optical  fiber  communication  can 
also  be  found  in  [10].  (Note,  though,  sometimes  we  emphasize  the  reliance  on  [9] 
by  explicitly  calling  out  the  reference.) 

We  wish  to  send  an  optical  pulse.  Receipt  of  the  pulse  is  considered  a  mark 
and  interpreted  as  the  symbol  1.  Non-receipt  of  the  pulse  is  considered  to  be  a 
space  and  is  interpreted  as  the  symbol  0.  If  this  were  all  there  was  to  it  (that 
is,  we  transmit  in  a  perfect  world)  we  would  have  a  discrete  memoryless  channel 
with  binary  input  and  binary  output  (a  (2,2)  channel  for  short).  The  physical 
process  of  sending  light  involves  various  mathematical  models,  physical  limitations 
and  inherent  noise.  Additionally,  the  mark  and  space  are  modeled  by  different 
probability  distributions. 

In  an  optical  fiber  channel,  light  is  sent  and  a  photodiode  at  the  detecting  end 
converts  the  light  into  a  voltage.  Since  the  photodetector  acts  ([9])  as  a  square- 
law  detector,  it  is  modeled  as  a  %2,  not  a  Gaussian,  distribution.  It  is  the  value 
of  the  voltage  that  all  decisions,  hard  and  soft,  are  made  against.  In  the  model 
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under  consideration,  it  is  possible  for  there  to  be  a  bit  error  in  what  is  received; 
the  transmitter  may  wish  to  send  a  0,  but  a  1  is  received,  or  it  is  desired  to  send  a 
1,  but  a  0  is  received.  There  is  also  the  possibility  of  symbol  ambiguity  where  the 
receiver  does  not  attempt  a  determination  as  to  what  was  sent — this  is  called  an 
erasure. 

The  most  general  model  is  the  binary  asymmetric  channel  with  asymmetric  era¬ 
sures  (BAC/AE);  see  Figure  1.  Figure  2  gives  guidance  as  to  how  a  symmetric  error 
probability  7,  and  the  erasure  probabilities  ao,ai,  relate  to  the  probability  densi¬ 
ties  of  sending  a  0  or  a  1.  The  simpler  binary  asymmetric  channel  (BAC)  model  is 
illustrated  in  [9];  this  is  a  (2,2)  channel.  In  the  BAC,  there  are  no  erasures,  and 
the  error  terms  are  not  equal  and  are  usually  taken  as  the  integral  of  the  distri¬ 
bution  for  sending  a  0,  between  tML  and  00,  and  the  integral  of  the  distribution 
for  sending  a  1,  between  0  and  tML-  The  value  Iml  is  where  the  two  distributions 
intersect  (we  will  return  to  this  later  with  a  caveat).  The  distributions  under  study 
are  single- humped,  so  tML  is  well  defined. 

Thus,  our  channel,  instead  of  being  a  (2,2)  channel,  is  a  (2,3)  channel.  In  [9] 
it  is  discussed  that,  through  various  assumptions  and  flexibility  with  respect  to 
the  soft  decision  thresholds,  it  suffices  to  study  a  binary  symmetric  channel  with 
asymmetric  erasures  (BSC/AE).  Note  that  the  BAC/AE  channel  with  soft  decision 
making  offers  higher  capacity  than  the  similar  BAC  [9].  Starting  with  a  BAC/AE 
channel  one  cannot  make  both  the  7 i  equal  and  the  ay  equal.  The  BSC/AE  channel 
model  makes  the  7 j  equal,  and  then  calculates  ao  and  a\  (we  return  to  this  later). 

In  a  BSC/AE  channel,  the  probability  that  a  transmitted  0(l)is  interpreted  as 
a  1(0)  is  7.  The  probability  that  a  transmitted  0(1)  cannot  be  decided  by  the 
receiver  and  is  taken  to  be  an  erasure  E  is  ao(ai).  They  are  determined  by  various 
voltage  values  (if  just  t-ML >  the  value  where  the  distributions  agree,  were  used  as 
a  cut-off  for  the  errors  with  no  erasures,  we  would  have  the  binary  erasure  (BAC) 
(2,2)  channel). 

Sending  a  0,  corresponds  to  a  “space.”  In  a  perfect  world,  there  would  be  no 
voltage  received.  However,  the  transmission  process  involves  sums  of  zero-mean 
Gaussian  random  variables,  plus  there  is  also  transmission  noise.  Due  to  this  the 
process  of  sending  a  0  is  modeled  as  a  central  y2  distribution,  where  the  degrees 
of  freedom  depend  on  the  dimensionality  of  the  transmitted  signal  space.  This 
probability  density  is  p(x |0),  with  x  being  the  received  value  conditioned  on  a  0 
being  sent. 

To  send  a  1,  which  corresponds  to  a  “mark,”  we  transmit  a  narrow  band  Gaussian 
process  that,  when  combined  with  signal  noise,  results  in  a  non-central  y2  distribu¬ 
tion.  This  probability  density  is  p(x\l),  with  x  being  the  received  value  conditioned 
on  a  1  being  sent. 

There  are  two  important  physical  constants  for  the  optical  fiber  communication 
channel  under  review  in  this  report.  The  constant  M  is  the  degrees  of  freedom  for 
the  underlying  y2  model.  The  constant  /3  is  the  signal  to  noise  ratio,  given  in  dB.  We 
have  /?  =  RcEb/No,  where  Rc  is  the  code  rate,  Ej,  is  the  energy  per  information  bit 
(that  is  a  mark),  and  Nq  is  the  power  spectral  density  of  the  amplified  spontaneous 
emission  noise. 

After  normalizing  for  the  various  physical  terms  in  the  optical  fiber  communi¬ 
cation,  via  the  constants  /3  and  M,  the  distributions  are  given  by  their  probability 
density  functions  as  [9] 
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(2-1) 


P{x  |0) 


xM  xe  x 
(M-  1)!  ’ 


and 


.M  —  1 

(2.2)  p(*|l)=(£)  2  (2y0)  , 

where  /(m-i)  is  the  modified  Bessel  function  of  the  first  kind  of  order  M  —  1. 

Our  goal,  as  in  [9],  is  to  calculate  the  capacity  of  the  BSC/AE  channel,  given 
values  for  M  and  /?,  against  the  symmetric  error  probability  7.  To  do  this  we  take 
a  7  value  and  (numerically)  determine  values  th  <  tML  and  t.R>  tML  by 


(2-3) 


7  = 


e~xe~l3h 


(M-l) 


(2 


dx  = 


r°°  xM-1e~x 


dx  . 


tR  (M-l)! 

Then,  once  we  have  the  tjJ  and  tR  values,  we  can  determine  the  erasure  proba¬ 
bilities  «o  and  a.\  by 


(2-4) 

(2-5) 


a0 


ai 


xM  1e  x 
(M-l)! 


dx 


M-l 

e-xe-Pl{M- 1) 


dx  . 


In  [9],  various  values  of  M  and  /?  are  discussed,  but  graphical  results  are  given 
only  for  M  =  3  and  /3  =  1, ...,  18.  Therefore,  we  restrict  our  analysis  to  the  same 
in  this  report. 

Note,  for  M  =  3  the  right  hand  side  of  Equation  (2.3)  simplifies  via  integration 
by  parts.  That  is,  since 


=  ^  {—  x2e  x  +  2  (— xe  x  —  e  x) }  +  constant  . 

We  have  that 

7  =  e— ((tjj)2  +  2t.R  +  2^j  , 

since  as  a;  ->  00,  e~x  — ►  0  faster  than  a  polynomial  of  a;  grows.  However,  this  still 
does  not  give  us  a  closed  form  for  tR,  so  we  may  either  numerically  integrate  or  use 
the  above  to  obtain  tR.  To  obtain  t^,  which  additionally  depends  on  /3,  we  must 
numerically  integrate.  Once  tR  and  are  obtained,  we  integrate  Equations  ( 2 .4) 
&  (2.5)  to  obtain  a0  an<l  ai- 
The  channel  matrix  is 


M  = 


-cto-7 

7 


01 0 

ai 


The  channel  input  random  variable  X  is  defined  so  that  the  probability  that  a  0  is 
sent  is  x  and  the  probability  that  a  1  is  sent  is  1  —  x,  and  let  the  output  random 
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variable  Y  takes  on  the  values  0,  E,  and  1.  Keep  in  mind  that  H(pi,  ...,pn)  = 
—  Y^i=iPi  1°§2  Pi  is  entropy  function  [8].  We  have  that 


P(Y  =  0) 


P(Y  =  0|X  =  0)x  +  P(Y  =  0|AT  =  1)(1  -  x) 
(1  -  a0  -  j)x  +  7(1  -  x) 

(1  -  a0  -  27)2  +  7  , 


P(Y  =  E ) 


P(Y  =  E\X  =  0)x  +  P(Y  =  E\X  =  1)(1  -  x) 
aox  +  «i(l  —  x) 

(ao  —  cti)x  +  Ot\  , 


P{Y  =  1) 


P(Y  =  1\X  =  0)x  +  P(Y  =  1\X  =  1)(1  -  x) 
jx  +  (1  —  ai  —  7)(1  —  x) 

(-1  +  +  2y)a;  +  1  —  «i  —  7  ,  so 


H{Y)  =  P(Y  =  0)  log2  P(Y  =  0)  +  P(Y  =  E)  log2  P(Y  =  E)  +  P{Y  =  1)  log2  P(Y  =  1)  . 


H(Y\X) 
H(Y\X  =  0) 
H(Y\X  =  1) 


H(Y \X  =  0)x  +  H(Y\X  =  1)(1  -  x)  ,  where 
ff(l  -  a0  -  7,a0,7)  ,  and 

if (7, or,  1  —  ai  —  7)  ,  and  the  mutual  information  is 


I(X,  Y)  =  H(Y)  -  H(Y\X)  .  Hence 

I(X,Y )  =  if((l  —  a0  —  2j)x  +  7,  (a0  —  Qi)x  +  «i,  (— 1  +  a\  +  2^)x  +  1  —  a\  —  7) 

-xH  (1  -  a0  -  7,  a0, 7)  -  (1  -  x)H  (1  -  -  7,  «i,  7)  . 


The  channel  capacity  [8]  is 

(2.6)  C  =  max/(X,  Y)  . 

X 


Keep  in  mind  that  we  are  dealing  with  a  (2,3)  channel.  For  a  (2,2)  channel, 
the  maximization  of  the  mutual  information  is  a  simple  calculus  problem  and  the 
problem  has  been  well  studied  (e.g.  [2,  5]).  However,  for  a  (2,3)  channel  there  is 
no  closed  form  solution  in  general  for  the  capacity.  Therefore,  numerical  techniques 
have  been  relied  on  for  the  capacity.  The  Blahut-Arimoto  algorithm  [4,  pp.  364- 
367], [3,  1]  was  used  to  numerically  determine  the  capacity. 

We  attempt  to  replicate  the  plots  given  in  [9].  As  noted  before,  our  plots  are 
somewhat  different  that  those  in  [9]. 
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3.  Plots 

In  this  section,  we  present  the  graphical  output  of  our  plots.  As  noted,  we 
offer  these  as  corrections  as  to  what  was  put  forth  in  [9].  We  note  that  in  [9]  the 
definition  of  the  test  condition  tML  was  slightly  adjusted  in  practice,  even  though 
suboptimal  detection  is  achieved,  from  how  it  was  theoretically  presented.  Ideally, 
as  discussed  above,  tML  is  the  non-trivial  x  value  where  p(x\o)  =  p{x |1).  In  [9],  for 
the  situation  under  consideration  in  this  report,  tML  is  actually  the  x  value  where 
fgML  p(x\l)dx  =  /t“  p(x\0)dx.  We  ran  our  plots  using  both  definitions.  The 
only  difference  is  that  the  second  definition  allows  slightly  larger  7  values,  there  is 
insignificant  difference  in  the  plots.  Therefore  we  stay  with  the  later  definition.  We 
ask  the  the  reader  to  keep  in  mind  that  the  only  use  of  tML  in  this  report  is  as  the 
test  condition  <  tML  <  t^. 


Capacity  (bits) 
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FIGURE  4:  Capacity  vs.  7  for  /3  =  1,  ...,6  and  M  =  3. 

Figures  4,  5,  &  6  show  the  plots  of  capacity,  for  M  fixed  at  3  and  /3  ranging 
from  1  to  18.  The  x-axis  is  the  value  of  the  error  probability  7  using  logarithmic 
scaling.  The  plots  are  of  capacity  vs.  7  for  M  =  3,  and  /3  =  1, ...,  18.  Even  though 
the  channel  matrix  is  given  in  terms  of  7,  a0>  and  07,  the  physical  interest  is  also 
in  seeing  how  capacity  depends  on  /3,  see  [9]. 

When  comparing  our  Figure  4  to  [9,  Fig.  15]  we  see  that  the  overall  shape  is 
the  same.  The  difference  is  that  our  maximums  are  larger  and  our  plots  decrease 
past  the  maximum. 

When  comparing  our  Figure  5  to  [9,  Fig.  16]  we  see  that  the  overall  shape  is 
the  same.  As  for  the  lower  f3  values  the  difference  is  that  our  maximums  are  larger 
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and  our  plots  decrease  past  the  maximum.  However,  when  comparing  our  Figure 
6  to  [9,  Fig.  17]  we  see  a  marked  difference.  As  /?  — >  18  our  plots  have  the  same 
shape  as  for  lower  value  /3.  However,  in  [9,  Fig.  17]  the  plots  approach  a  constant 
value  of  1,  as  /3  grows.  We  attribute  the  differences  to  numerical  errors  in  [9].  We 
offer  theoretical  reasons  why  our  plots  are  reliable  below. 


-10  -8  -6  -4  -2  0 

log10(y) 


FIGURE  5:  Capacity  vs.  7  for  (3  =  7, ...,  12  and  M  =  3 
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FIGURE  6:  Capacity  vs.  7  for  /3  =  13, ...,  18  and  M  =  3 


4.  Capacity  Behavior  and  Future  Research 

It  is  interesting  to  see  that  capacity  behaves  in  a  slightly  humped  manner.  On 
a  logarithmic  scale,  the  capacities  start  off  near  0  for  very  small  7.  The  larger  /3, 
the  smaller  7  is  until  the  capacity  is  essentially  0.  As  7  — >  0+,  both  ao  and  or 
approach  1,  so 


hence  C  — >  0  as  the  plots  show. 


M 


0  1  0 
0  10 
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The  capacity  increases  as  7  grows,  and  then  slightly  decreases  as  7  grows  to  its 
upper  limit.  Specifically,  capacity  decreases  and  falls  off  slightly  as  7  -+«  10-'43  , 
for  /3  =  1,  and  upper  limiting  to  ss  10-2  ,  for  /3  =  18.  The  upper  limits  of  7  are 
determined  by  the  requirement  <  Iml  < 

We  wish  to  investigate  why  the  plots  seem  to  peak  just  shy  of  their  upper  limits, 
and  the  physical  implications  of  this.  There  is  no  closed  form  solution  for  the 
capacity  of  a  generic  (2,  3)  channel,  but  we  would  still  like  to  glean  some  knowledge 
of  the  capacity  behavior  via  direct  examination  of  the  channel  matrix.  For  a  (2,  2) 
channel,  this  approach  closely  tracks  the  behavior  of  the  capacity.  Specifically,  in 
[6],  it  is  shown  that  the  capacity  C’2,2  for  a  (2,  2)  channel  with  channel  matrix 


obeys 

(4.1) 


1 

1 


2 ln(2) 


+  (^2,2  < 


b  I 


This  result  has  been  generalized,  and  for  a  (2,  3)  channel  with  channel  matrix 


M  = 


a2  a\ 
62  b) 


it  has  been  shown  in  [7]  that  the  capacity  C2  3  is  bounded  as: 

(4.2) 

1 

(|«i— 6i|  +  |a2  — 62I  +  IS— b\)2  <  62,3  <  max(ai+o2,  61+62)— min(ai,  61)— min(a2, 62)- 


i  In  (2) 


We  hope  to  use  Equation  (4-2),  along  with  more  information  on  how  a0  and  ay 
depend  on  7  to  attempt  to  derive  rule-of-thumb  capacity  results  for  the  finer  optical 
communication  channel  under  study  in  this  report. 
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